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Abstract 


Embedding support devices in educational computer games has been asserted to positively 
affect learning outcomes. However, there is only limited direct empirical evidence on which 
design variations of support provision influence learning. In order to better understand the 
impact of support design on novices’ learning, the current study investigates how support 
devices and their type of provision (intrinsic vs. extrinsic) determine games’ effectiveness 
on learning outcomes. This effectiveness is also related to how the design-type of provi¬ 
sion influences learners’ virtual presence and cognitive load. Compared to an educational 
adventure game without additional support, the results indicate that the game equipped 
with support devices enhances learning outcomes, although no differences in cognitive load 
were found. A variation in the design of provision shows no effect. In order to gain a more 
thorough understanding of support devices and their design for games, additional learner 
characteristics (e.g., interest) should be considered in future research. 
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Introduction 


Instructional approaches based on discovery (Bruner, t96r), experiential (Kolb, 1983), or 
problem-based learning (Barrows & Tamblyn, t98o) as well as constructivist ideas (Jo- 
nassen, t99t) stress the importance of learning environments such as educational com¬ 
puter games. They are seen as opportunities to enhance learning and motivation by of¬ 
fering learners open-ended, autonomous learning. As they permit learners to discover or 
construct learning information themselves, theorists consider them to be a promising al¬ 
ternative to the presently more common approach of presenting learners with informa¬ 
tion for example by listening to a teacher (Kirschner, Sweller, & Clark, 2006). Although 
the beneficial effects of autonomous learning in educational games are supported from a 
theoretical perspective, empirical research shows both beneficial and detrimental effects on 
learning outcomes (e.g., Beale et al., 2007; Dede et ah, t997, 2005; Parchman et ah, 2000; 
Wong et ah, 2007). While some learners benefit from learning in educational computer 
games, for other learners, especially those with little prior knowledge, educational games 
have proven no more effective than traditional media (for an overview on games’ learning 
effectiveness see Schrader, 20to; Schrader & Bastiaens, in press). According to Kirschner 
et ah (2006), one of the potential threats not only of educational computer games but of all 
discovery-based, problem-based, or experiential learning environments is that they make 
heavy demands on learners’ working memory capacity by requiring unnecessary cognitive 
load that exceeds the limitation of working memory. In this case, learning will be inhib¬ 
ited (Sweller, t993). Following the line of cognitive load theory (Chandler & Sweller, 1991; 
Sweller, f994,1999), the total cognitive load imposed on working memory during learning 
is additively composed of three load types: intrinsic cognitive load, extraneous cognitive 
load, and germane cognitive load. Intrinsic cognitive load depends on the complexity of the 
given task in relation to learners’ level of expertise. Extraneous cognitive load depends on 
the instructional design of learning material; whereas, germane cognitive load results from 
learners’ engagement in the learning activity. During learning with games, an example of 
intrinsic and extraneous factors that overload learners’ processing resources is the amount 
and complexity of information that needs to be processed, combined with simultaneous 
actions of cognitive and motor activities (Kalyuga & Plass, 2009; Kerres, Bormann, & Ver- 
venne, 2009; Lim, Nonis, & Hedberg, 2006; Whitton, 20to). Learners have to give high 
attention to how to manipulate the game and control experimental tasks through motor 
actions. In addition, they have to search for the tasks that perhaps are delayed in space 
and time in the gaming environment. These are often more complex than simply chatting, 
shooting, or regulating experimental simulations, and the completion of the objective may 
involve several or all of these. The learning objectives of tasks are not obviously given, but 
gradually unfold while learners interact with the game to complete them. They have to dis¬ 
cover and analyze what the learning objectives behind these tasks are. They have to draw 
hypothetical conclusions from the result of their attempts to solve the task, especially when 
they have failed to reach their goal and need to repeat the attempt. 

Thus, learning with games seems to pose a challenge even to learners with a high level of 
expertise in the learning topic, and more so to novices. Martens et al. (1997) suggest that 
learners with higher developed prior knowledge on the learning topic have a better base for 
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compensating for complexity, intransparency, and incoherence within information. They 
have knowledge structures in long-term memory available, which help to organize and link 
relevant information. In contrast, novices have no sufficient prior knowledge and corre¬ 
sponding knowledge structures in their long-term memory to compensate for ineffective 
activity. The provided tasks in games are often too difficult and mentally too demanding 
for novices. They prevent them from accurately processing the necessary information. As 
a result, novices might experience a total cognitive load that exceeds their limited working 
memory capacity and leaves them with insufficient cognitive resources for successful learn¬ 
ing. Novices could also revert to less meaningful learning to keep their cognitive load within 
the threshold limit (Aleven et ah, 2003). This attempt to avoid cognitive overload through 
unselective, explorative modes of information processing is as detrimental to learning as a 
cognitive overload itself. 

For novices, providing support devices might be one possible approach to reducing cogni¬ 
tive stress in terms of task complexity and difficulty. The effectiveness of this approach is 
supported by research in the field of cognitive load theory (e.g., Chall, 2000; Kirschner, 
Sweller, & Clark, 2006; Klahr & Nigam, 2004; Tuovinen & Sweller, 1999) but also by theo¬ 
retical frameworks of classroom teaching (e.g., Klauer, 1985). Moreno (2004) concluded 
from a literature review of studies comparing pure discovery learning with guided forms of 
learning that “the debate about discovery has been replayed many times in education but 
each time, the evidence has favored a guided approach to learning” (p. 18). Game research¬ 
ers (e.g., Cobb & Fraser, 2005; Leemkuil, 2006; Leutner, 1993; Rieber, Tzeng, & Tribble, 
2004; Salzman et ah, 1999; Standen et ah, 2001) therefore stress the importance of support 
devices. They may overcome difficulties of task complexity by presenting essential informa¬ 
tion. They help make the uncovering of relations between the provided learning objects 
more explicit and transparent (de Jong, 2005). Also, they aim at cognitive processes of 
learners by directing learners’ attention to the essential learning information. Therefore, 
combined with an adequate game design, they may reduce information search and, con¬ 
sequently, cognitive load in order to enhance the learning outcomes (de Jong, 2005), that 
is the successful restructuring and integration of new information in existing knowledge 
structures (Krapp & Weidenmann, 2001). But support devices not only seem to enhance 
learning outcomes through preventing cognitive overload, they may also strengthen the re¬ 
lation between learners’ experience of virtual presence and their learning. Virtual presence 
is defined as an actual subjective emotion-related state in which a user is fully immersed in a 
virtual activity provided by technological means. For games, their positive impact on virtual 
presence was affirmed in several studies (e.g., Heers, 2005; Welch et ah, 1996). Further¬ 
more, in the study carried out by Schrader and Bastiaens (2012), it was demonstrated that 
virtual presence is positively associated with learning success. Adapted from research on 
emotions in the fields of psychology, education and computer science (e.g., Dweck, 2002; 
Lepper & Henderlong, 2000) and education (e.g., Ainley, Corrigan, & Richardson, 2005; 
Meinhardt & Pekrun, 2003; Meyer & Turner, 2002; Pekrun et al., 2002), it was argued that 
this result is based on virtual presence’s function to motivate and stimulate learners to in¬ 
vest mental effort in learning with games. Also, virtual presence guides learners’ attention 
away from the interface towards the gaming content. This increases the cognitive capacity 
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actually allocated to the learning task. However, the findings of the study also showed that 
virtual presence does not guarantee better learning outcomes. Besides learners’ individu¬ 
al tendency to invest in virtual presence, it was shown that a heavy cognitive load, which 
might be due to inadequate game design and complexity of tasks, reduced virtual presence 
and its positive impact on learning (Schrader & Bastiaens, in press). 

Because of the described benefits of support devices on cognitive load, virtual presence, and 
learning outcomes, game designers do not always ignore the role of support. An analysis of 
current support devices in educational computer games reveals a variety of designs (e.g., 
direct vs. non-direct, game-extrinsic vs. game-intrinsic, before vs. during game-playing, 
learner-requested vs. automatically system-initiated). All are based on the idea of support¬ 
ing learners by giving information on the learning topic or gaming behaviour. However, the 
few existing studies on support devices and their relation to learning outcomes (e.g., Leem- 
kuil, 2006; Leutner, 1993; Nelson, 2007) show that even if support is provided it does not 
always improve learning outcomes, especially not if learners have a low level of expertise 
(Nelson, 2007). In accordance with these results, this article argues that support devices 
are more effective predictors of learning success if they are designed to reduce cognitive 
load while also enabling virtual presence. Hence, the current article focuses on support 
devices designed to mediate the embedded learning topic effectively rather than to instruct 
the learner on how to interact with the gaming environment. 

Considering this background, the area of particular interest in this study is on the design 
of support devices in terms of their types of provision, more specifically on two different 
types: intrinsic versus extrinsic support. In particular, this study investigates which one of 
these two types is most capable of maximizing virtual presence and minimizing cognitive 
load to advance learning outcomes. The type of provision is interesting since advocates of 
pure discovery learning oppose not only integrating content support into the gaming envi¬ 
ronment but also take different stances on this issue (e.g., Hofer et al., 1996; Nelson, 2007; 
Renkl, 2002; Wood, 2001). Besides, especially in the field of educational computer games, 
consolidated findings are rare. 

Therefore, the following section discusses the advantages and disadvantages of the dif¬ 
ferent methodological approaches in the areas of support provision with regard to virtual 
presence, cognitive load, and learning outcomes. 


The Provision of Support Devices: Intrinsic versus Extrinsic 


Based on the basic support categories of Gery (1995), an analysis of current support devices 
in computer games shows that support can be presented in an intrinsic or in an extrinsic 
way. Intrinsic support automatically provides learning information during game-playing 
that is adequate to the learners’ actual needs, that is when learners make a mistake (e.g., in 
Winterfest [Alphabit, 2012]). In contrast, extrinsic support is not part of the gaming world. 
It offers domain-specific background information such as definitions, basic facts, and con¬ 
cepts mostly presented in the form of a hyperlinked textbook (e.g., in Bioscopia [Heureka- 
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Klett, 2011]; River City [Dede, Ketelhut, & Reuss, 2003]). 

Hofer et al. (1996) suggest that the intrinsic type of support provision - if it is adequately 
designed (for an overview of design see e.g., Narciss, 2006) - may be more effective for 
learning compared to an extrinsic hyperlinked textbook. While the authors did not elabo¬ 
rate on possible causes for the learning ineffectiveness of extrinsic support, the effects they 
observed can be explained with a decrease in virtual presence and an excessive cognitive 
load. It can be argued that switching between the gaming environment and the extrinsic 
hyperlinked textbook not only causes a break in the experience in terms of virtual presence. 
It also requires additional working memory resources unrelated to learning and thereby 
overloads the limited working memory capacity. Whereas the intrinsic support provides 
explicit information that helps to continue solving a gaming task, information given in the 
extrinsic type of support does not necessarily help to make judgements about the actual 
task in the game. Here, learners have to pause their game-playing for searching of suitable 
information and have to adapt it to the current task inside the game. Especially for novices, 
who may not understand or misinterpret the learning purpose of the tasks, it is practically 
impossible to select the relevant information for a specific learning task at hand. Conse¬ 
quently, they are not likely to profit from extrinsic support in terms of learning. 

Besides their difference in spatial presentation and alignment on given tasks, both types 
also differ in their degree to which the control of support is given to the learners. Whereas 
intrinsic support is volunteered by the game-system, extrinsic support has to be invoked 
from the learner. This influences not only virtual presence, cognitive load, and learning 
outcomes but also the use of help. As highlighted in several studies (e.g., Hofer et al., 1996; 
Jiang, Elen, & Clarebout, 2009; Nelson, 2007; Wood, 2001), the effect learners’ control of 
support has on learning outcomes may not always be materialized as some researchers stat¬ 
ed (Hofer et al., 1996; Leutner, 1993; Renkl, 2002). Nelson (2007), for example, evaluated 
the influence of learner-requested access to support in a learning simulation, but found 
no measurable impact on learning. He notes that an important feature of support is that 
learners are given the choice of whether or not to use it. When support devices are offered 
for optional use, as in the type of extrinsic support, learners reported lower learning out¬ 
comes because instead of using the support device they tried to guess the right answer. In 
addition, Wood (2001) found that novices are the least likely to use support appropriately 
when it is under their own control. It was argued that novices have weaker help-seeking 
behaviour or that they might be unable to cope with the demands of searching for help. 
They are faced with tasks, which are, given their low prior knowledge, subjectively already 
difficult enough. Thus, the learning effectiveness of extrinsic support is minimal especially 
for novices because they do not actively use help as appropriate. Hence, novices have to be 
provided automatically with intrinsic support, continuously added as learners repeatedly 
fail at solving a task. This strategy of information delivery would guarantee the appropriate 
use of the help function by offering the novices assistance tailored to their actual need in 
a specific situation and, at the same time, prevent overloading the learners’ cognitive re¬ 
sources and preserve learners’ virtual presence by allowing them to continue concentrating 
on the current task. 
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Research Questions 


For support devices to be effective in games, the above review suggests that they must be 
designed to optimize virtual presence, cognitive load, and learning outcomes. Otherwise, 
support devices may hamper learning. The design of support in terms of their provision 
should help to meet these requirements. The present research therefore investigates under 
which type of provision support is most effective in relation to learners’ low prior knowl¬ 
edge of the learning topic. In particular, this study addresses the following two research 
questions. 

(1) Are virtual presence, cognitive load, and learning outcomes different for novices who 
use educational computer games with support devices compared to novices who use an 
educational computer game without support? 

According to this first research question, for virtual presence it is expected that a game 
without additional support devices generates a higher degree of virtual presence in contrast 
to both gaming variations with support because game-playing is not interrupted due to 
offering any kind of help. For cognitive load, it is assumed that both game versions with 
support decrease cognitive load by providing essential learning information in compari¬ 
son to an educational computer game without support. Support in general might reduce 
the search for information and directing working-memory resources to the actual intended 
learning activity. As a result, both versions with support should increase learning outcomes 
more than a conventional game version without any support. 

(2) Are virtual presence, cognitive load, and learning outcomes different for novices who 
have access to intrinsic support from those of novices who have access to extrinsic support? 

With respect to this second research question, it is expected that intrinsic support enables 
a more intensive feeling of virtual presence and avoids high cognitive load in contrast to 
extrinsic support. As discussed in the theoretical part of this article, this assumption is due 
to the spatial split-attention effect brought about by switching between the game and the 
hyperlinked-textbook. Moreover, it is expected that game-internal support presents infor¬ 
mation immediately required to deal with the current task in the game. As a result, intrinsic 
support is expected to promote learning outcomes more than extrinsic support. The differ¬ 
ence in learning outcomes between both support conditions is not only based on the fact 
that extrinsic support causes a heavy cognitive load when used. Also, lower learning out¬ 
comes can be the result of underusing them by learners, even if they require help. 

To sum up, the effectiveness of support in games for learning outcomes is hypothesized 
since it encourage learners to use it and preserves, rather than disrupts, learners’ experi¬ 
ence of virtual presence. In addition, support devices should reduce cognitive load by im¬ 
plicitly providing help that is adaptive to learners’ needs. 
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Method 


Participants 

One hundred and thirty-five 8 th graders (65 male and 73 female; mean age = 13.44 years, SD 
= .57) of the preparatory high school Ricarda-Huch Gymnasium in Hagen, Germany par¬ 
ticipated in this study. As this study examines the effect of support devices on novices, the 
researchers intentionally selected only learners that were not familiar with the embedded 
learning topic in the game. Thus, none of the participants had prior knowledge of the learn¬ 
ing topic. In addition, participants were not familiar with the educational computer game 
used in the study. Learners were randomly assigned to one of the following three versions 
of an educational computer game: (1) without additional support (G) (n = 43), (2) with ex¬ 
trinsic support (G_ES) (n = 47), and (3) with intrinsic support (G_IS) (n = 45). 

The Gaming Environment 

The computer-based educational game Elektra (European commission, 2009) was adapt¬ 
ed for this experiment. It offers a coloured, three-dimensional (3D) single-user adventure 
game for learning physics content for 8 th graders. The gaming scenario takes place in an old 
house in Florence. From a first-person view, learners can move freely through the house’s 
rooms, explore, and manipulate objects to solve learning problems presented to them. They 
gather information and perform a series of three interactive experimental tasks, through 
which they can learn, for example, about the concept of light refraction. 

For the study, three versions of Elektra were developed: (1) without support (G), (2) with 
extrinsic support (G_ES), and (3) with intrinsic support (G_IS). 

The first design condition, used as the control condition, is the conventional design of a 
gaming environment. This design follows the tenet that learners should construct knowl¬ 
edge completely on their own by discovery learning, a situation that requires learners to 
solve the provided tasks without any support. Learners have to learn about light refraction, 
that is that light consists of rays and how it can be focused. The learner has to send a strong 
beam of light through the keyhole of a door in order to open it. Behind the door, the beam 
has to hit a light cell but not anything else. The learner can use a virtual lab with a flashlight 
and blinds to learn how to create a narrow beam of light and transfer it to the light cell to 
open the door. If the learner fails in solving the given tasks, no prompts on how to proceed 
are presented. The learner has to keep Lying until the correct solution is discovered (see 
Figure 1). 
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Figure l. Conventional design of Elektra without support. 


The second design condition (G_ES) is similar to the first in that no support is given. Rath¬ 
er, descriptions of the relevant learning concepts are presented in a separate hyperlinked 
textbook. The learners are made aware of the support device that is always available by a 
permanently visible button with a hyperlink to the textbook (see Figure 2). 
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Figure 2. Design of Elektra with extrinsic support in the form of a hyperlinked textbook. 


In contrast, the third design condition (G_IS), featuring intrinsic support, provides dynam¬ 
ically embedded support whenever necessary. The system offers support via a pedagogical 
agent that gives simple hints to encourage learners to keep trying if they have made an 
error. If a learner repeatedly fails to solve a given learning task, the system provides com¬ 
ments and explanations that contain the correct answer (see Figure 3). 
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Figure 3. Design of Elektra with intrinsic support in the form of a pedagogical agent. 


Measurement Instruments 


Background questionnaire. 

Before the game-playing experiment started, a background questionnaire was used to gath¬ 
er data on prior knowledge, that is prior knowledge of learning topic physics (3 items), 
game literacy (1 item), age (1 item), and gender (1 item). 

Virtual presence questionnaire. 

Virtual presence was measured using Witmer and Singer’s Presence Questionnaire (PQ) 
(1998) that consists of 10 items. Learners rated their virtual presence on a 5-point Likert 
scale ranging from 1 (totally disagree ) to 5 (totally agree). The PQ measures to what extent 
learners experienced the feeling of being involved in the gaming environment. The indi¬ 
cators used are the degrees to which they are able to fade out distractions external to the 
gaming environment, feel in control, and experience their virtual interactions and move¬ 
ments as natural. Due to a low item-scale test correlation one item was not included in the 
analysis. 
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Cognitive load questionnaire. 

Cognitive load was measured with eight items on a 5-point Likert scale ranging from 1 
{very easy ) to 5 {very difficult ). One item is the Paas and van Merrienboer (1993) item that 
is often used for measuring cognitive load in terms of invested mental effort, that is how 
much mental effort the learner had to invest to complete the learning tasks in the gaming 
environment. The other seven items are the Kalyuga and Paas (2005) items for measuring 
perceived levels of difficulty to assess the specific demands on working memory (intrinsic, 
extraneous, germane load) that resulted from the different requirements of the educational 
environments or the learning content within them. Five of the eight items were significant 
with the cognitive load total score (.52 <r it > .73). This indicates that it was not possible to 
separate different aspects of cognitive load affected by different specific demands of the 
virtual environments and learning tasks within. Thus, the mean of all items as an indicator 
of overall cognitive load in the data analyses was applied. 

Learning outcome questionnaires. 

Outcomes concerning the learning topic were measured to assess the differences in reten¬ 
tion, comprehension, and near and far transfer. After the questionnaire was constructed, 
two teachers of physics were asked to review it. Both were assessing the questions in re¬ 
gards to their readability and if they would be understandable and clear for learners at 
the educational level of eighth graders. Where necessary, the questionnaire was adapted 
according to their review. 

Retention of facts and concepts as a more trivial learning outcome was measured by five 
multiple-choice questions. Each question was scored with one or zero points according to 
whether it was answered correctly or incorrectly. The quality of comprehension was mea¬ 
sured with four open-ended questions, in which the learners had to demonstrate their un¬ 
derstanding of the acquired knowledge in their own words. Here, the maximum test score 
was one point for each question. The transfer tasks comprised six near-transfer tasks and 
four far-transfer tasks to measure the quality of the transfer of the acquired knowledge to 
new situations (for a discussion on differentiating transfer, cf. among others Hasselhorn & 
Mahler, 2001). The near-transfer tasks were analogous to the learning objects in the game, 
but contained different objects than those presented in the game (e.g., a match instead of a 
wooden ball). The far-transfer tasks were different in structural features and were meant to 
determine whether or not learners were able to apply the acquired knowledge to unknown 
and more complex situations. For example, a task on magnetism required the learners to 
explain why one cannot use buttons the size of a coin instead of real coins to pay a parking 
fee at a ticket machine. Here, the maximum test score was eight points. Due to a low index 
of task complexity {p < .20), one item for retention, near transfer and far transfer was not 
included in the analyses. 

Apart from learning outcomes, game knowledge was measured with 10 questions testing 
knowledge about the storyline and technical features. 

Table 1 shows the scales of the questionnaires, the number of items per scale accompanied 
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by an example of an item, and the reliability coefficients of all scales. 
Table l 

The Scales of Questionnaire and their Reliability 


Main scale 

Items Example 

Reliability 

Prior Knowledge 

4 

My knowledge concerning light refraction is ... 

•59 

Virtual Presence 

9 

I felt that I was able to control events in the virtual environment 

f Involvement and control ) 

.86 



I felt like I was immersed and interacting in the virtual environ¬ 
ment ( Involvement and control) 




While working in the virtual environment I was able to fade out 
external distractions ( Distraction ) 


Cognitive Load 

8 

When learning with the virtual environment I invested very 
low...very high mental effort ( Mental effort ); 

.82 



The physical learning topic was ... (Intrinsic load); 




Working with the virtual environment was... (Extraneous load); 




Understanding the physical learning topics and the relation be¬ 
tween the three major topics were ... (Germane load) 


Learning Outcomes 




Retention 

4 

Which objects do magnets attract? Please choose the objects! 

•59 

Comprehension 

4 

Why do you think that magnets attract the objects? Please give 
reasons for your choice! 

.60 

Near Transfer 

5 

Which objects do magnets attract? Please choose the objects! 

.61 

Far Transfer 

3 

Why can’t one use buttons the size of a coin instead of real coins 
to pay the parking charge of a car? 

•59 

Game Knowledge 

10 

Which button does one have to press to get Galileo’s support? 

.66 


Procedure 

The study was conducted in the computer labs of the FernUniversitat in Hagen, Germany. 
At the beginning, all learners participating in this study took a self-reported knowledge test 
about the physics learning content and about the game. Also, demographical data of the par¬ 
ticipants were collected. Afterwards, learners were randomly assigned to one of the three 
experimental conditions. Subsequently, the learning phase started, which took one hour ap¬ 
proximately. Directly after the exploration, learners filled out the virtual presence and cogni¬ 
tive load questionnaires (15 minutes). These questionnaires were followed by the learning 
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tests (40 minutes). 

Analysis 

One-factorial analyses of variance (ANOVA) were performed to measure the overall differ¬ 
ence in the mean level of virtual presence, cognitive load, and learning outcomes between 
the groups presented with three different design conditions of the same educational com¬ 
puter game (G, G_ES, G_IS). 

As the overall differences between the three groups was significant, analyses of contrasts 
were conducted to carry out two comparisons: first, between the group who learned with 
the G version on the one hand and both groups who learned with the support versions 
(G_ES, G_IS) on the other hand and, second, between both groups with support (G_ES, 
G_IS). Analyses of homogeneity of variances indicated that for all groups the population 
variances were equal. Also, all scores were normally distributed. Thus, it was shown that 
the use of ANOVA and contrast analyses was justified. For all statistical tests, a significance 
level of .05 was maintained. In addition to the statistical significance, for ANOVA Eta 2 and 
for contrast analyses r , ontrast was used as the degree of the effect size showing the practical 
significance. 


Results 


The additional analysis of the G_ES group’s utilization of the hyperlinked textbook indi¬ 
cates, as was hypothesized, that the textbook was accessed infrequently. Only 30 learners 
used the linked textbook for help during game-playing, whereas 17 learners reported that 
they had ignored it. In order not to adulterate the results concerning the effectiveness of 
support devices on virtual presence and cognitive load, the analysis for the G_ES group in¬ 
cludes only the data of the 30 learners who used the textbook. The number of learners that 
dropped out was evenly distributed over the conditions ( X 2 = .29, p = .95). This resulted 
in the following group composition: (1) game without additional support (G) (n = 30), (2) 
game with extrinsic support (G_ES) (n = 30), and (3) game with intrinsic support (G_IS) 
(n = 31)- 

There was no statistically significant difference between the three groups concerning gen¬ 
der: (1) male = 13, female = 17; (2) male = 15, female = 15; (3) male = 14, female = 17; X 2 = 
.10, p = .94. Also, the collected data for determining learners’ prior knowledge showed no 
differences between learners {F{ 2, 88) = .33, p = .71). As well, participant comparability 
with respect to the learning topic was assured because the presented learning topic of phys¬ 
ics is embedded in the school curriculum for eight graders of Gymnasium - the German 
equivalent of the British grammar school or US preparatory high school - and all partici¬ 
pants were only at the beginning of their eighth year. In addition, learners had no previous 
experience of the used educational computer game Elektra. Thus, these results ruled out 
the possibility of a prior knowledge effect connected to the given learning topic or the game. 

The mean results for virtual presence, cognitive load, and learning outcomes are summa- 
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rized in Table 2. 

Table 2 

Means (and Standard Deviations) for Virtual Presence, Cognitive Load, and Sum Scores 
for Learning Outcomes of the Three Groups 



G 

G_ES 

G_IS 


(n = 28) 

(n = 30) 

(n = 31) 


M {SD) 

M (SD) 

M (SD) 

Virtual Presence 

3.90 (.80) 

2-95 (- 76 ) 

3.56 (. 84 ) 

Cognitive Load 

3.71 C82) 

3-56 (.86) 

3.44 (.76) 

Learning Outcomes 




Retention 

.82 {. 54 ) 

1.69 (.98) 

2.00 (1.01) 

Comprehension 

1.28 (.58) 

1.96 (.81) 

2.20 (.98) 

Near Transfer 

2.60 ( 1.44) 

3-70 (. 95 ) 

3.78 (1.16) 

Far Transfer 

.78 (.24) 

1.20 (.32) 

1.24 (.80) 

Game Knowledge 

9-57 (1-83) 

9.50 (2.08) 

9.50 (2.30) 


Virtual Presence 

Concerning virtual presence, the means in Table 2 shows that the G group obtained a mean 
score of 3.90 (SD = .80), the G_ES group a mean score of 2.95 (SD = .76), and the G_IS 
group a mean score of 3.56 ( SD = .84). These differences were statistically significant (F( 2, 
88) = 11.03, P < - 05 > n 2 = .20). Additionally, separate analyses of contrasts confirmed the 
general hypothesis that the G group differed in its virtual presence from both groups that 
were provided with support. The conventional version of the educational computer game 
without support devices increases virtual presence significantly above the levels of virtual 
presence seen in both groups that used support versions (t( 88) = 3.52, p < .05, r = .35). 
Moreover, a significant effect was found for the comparison between both groups with sup¬ 
port devices (f(88) = 3.00, p < .05, r = .30), that is the rated feeling of virtual presence was 
higher in the G_IS group than in the G_ES group. 

Cognitive Load 

According to the means, the G group reported the highest mean of cognitive load, whereas 
the G_IS group demonstrated the lowest mean of cognitive load during game-playing. The 
G_ES group’s reported mean was at the centre. The results of the one-way ANOVA to check 
the statistical differences among the groups, however, demonstrated that there is no signifi¬ 
cant difference in cognitive load among the groups (F( 2,88) = .77, p = .46). 
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Learning Outcomes 

Differentiating the learning outcomes for the learning topic of physics in terms of reten¬ 
tion, comprehension, and near and far transfer, the G group reported the lowest sum scores 
in learning outcomes compared to the G_ES group and the G_IS group. The difference in 
sum scores between all three groups was statistically significant for retention, comprehen¬ 
sion, and near transfer (for retention: F( 2,88) = 11.17, P < ■ °5 > r2 = -20; for comprehension: 
F( 2,88) = 5.65, p < .05, n 2 = .11; for near transfer: F( 2,88) = 8.78, p < .05, n 2 = .16), but not 
for far transfer (h'(2,88) = 1.44, p =. 24 ). In addition, analyses of contrasts showed that the 
G group differed in these learning outcomes from both groups that were provided with sup¬ 
port (for retention: f(88) = 4.59, p < .05, r = .43; for comprehension: f(88) = 3.27, p < .05, r 
= .32; for near transfer: t(88) =4.17, p < .05, r = .40). The contrast analysis of the compari¬ 
son between the G_ES group and the G_IS group demonstrated nonsignificant effects (for 
retention: t(88) = 1.20, p = .22; for comprehension: f(88) = .85, p = .39; for near transfer: 
t(88) = .29, p = .77; for far transfer: f(88) = 2.74 ,p = .07). 

For game knowledge, the results reported a contrary effect than for the outcomes of the 
learning topic: Although the difference in sum scores of gaming knowledge was not signifi¬ 
cant between groups (F( 2,88) = .01, p =.98 ), the differences in the sum scores nevertheless 
demonstrated that learners in the G group reported higher game knowledge than did learn¬ 
ers who learned with the G_ES version and learners who learned with the G_IS version. 


Discussion and Conclusion 


Based on criticism of tenets of pure discovery, problem-based, and experiential learning, 
this study compared the effects of educational computer games with different designs of 
support devices. The study focussed on computer games with intrinsic or with extrinsic 
support. The study measured the influence the type of support system has on virtual pres¬ 
ence, cognitive load, and learning outcomes and compared these results to the influence of 
conventional educational computer games without any support devices. 

Results show that the effectiveness in terms of virtual presence was influenced both by 
the presence of support devices in educational computer games and their design as well. 
As expected, learners who learned with the conventional educational computer game re¬ 
ported the most intense experience of virtual presence because they were not interrupted 
in game-playing by support devices. The comparison of both groups using support devices 
demonstrates that the group who learned with the educational computer game with intrin¬ 
sic support reported a more intensive virtual presence than the group provided with the 
support device in the form of an extrinsic hyperlinked textbook. This result confirms the 
hypothesis that virtual presence is diminished by the break in learners’ game-playing in 
order to search for support in the hyperlinked textbook. According to these results, support 
devices are more effective in terms of virtual presence if they are intrinsic, that is integrated 
into the gaming environment. 

Moreover, intrinsic support was hypothesized to have a positive influence on learning out- 
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comes not only by enhancing virtual presence, but also by avoiding heavy cognitive load. 
However, for cognitive load, the theoretical hypothesis was not confirmed. All three game 
conditions do not significantly differ in affecting cognitive load. This result shows that sup¬ 
port devices can affect the demand on cognitive processes not only in a positive, but also 
in a detrimental way. This might support the idea that even if support provides useful in¬ 
formation for solving the given task, it also causes cognitive effort for novices, who have to 
deal with both the task and the help simultaneously. This explanation, in addition to the 
fact that cognitive load is neither affected by the use of support devices nor by their imple¬ 
mentation, also accounts for the observation that not all learners used learner-requested 
extrinsic support. In accordance with the theoretical assumption that especially novices do 
not use support devices because they might be unable to cope with additional information 
(Wood, 2001), the additional analyses show that 17 of 47 learners ignored the extrinsic 
support device. It can be argued that these learners refrain from requesting support in or¬ 
der to avoid cognitive load and to preserve their virtual presence and involvement in the 
gaming environment. Whereas Aleven et al. (2003) suggest that an appropriate design of 
support might compensate for the cognitive load of its use, in our study the cognitive load 
was not reduced by the support device designed to eliminate additional cognitive load by 
integrating the help into the gaming environment. A reason for this finding could be due 
to the design characteristics of the given extrinsic support. The possibility of stopping the 
game to search for help outside (i.e., not doing game-playing and help-searching simul¬ 
taneously) might have prevented the expected cognitive load. While the results discussed 
suggest no significant difference in cognitive load between all three game-design versions, 
support seems to be effective in terms of learning outcomes. Based on this result, a straight¬ 
forward practical implication of this study is that support should be provided within edu¬ 
cational computer games. Learners with access to support, regardless of its design, showed 
significantly greater gains in test scores for retention, comprehension, and near transfer 
compared to those with no access to support in the conventional gaming condition. For far 
transfer as a nontrivial and more complex learning outcome, the hypothesis of improved 
learning outcomes due to support in general could not be confirmed. Nevertheless, given 
the low test score for learning outcomes in terms of retention, comprehension, near and far 
transfer, any interpretation on the basis of learning effects should be considered with cau¬ 
tion. Contrary to the significant differences between the group presented with the conven¬ 
tional educational computer game on the one hand and both groups with support devices 
on the other, the differences in design between both support devices did not have any mea¬ 
surable effect on learning outcomes in terms of retention, comprehension, and near trans¬ 
fer. Dochy and Segers (1997) provide a possible explanation for this finding. The authors 
hold that in classroom practice a high level of interest strengthens the relation between 
prior knowledge, the use of support, and learners’ mental effort. Adapting this result to our 
study, it can be assumed that the group using the extrinsic support device had a higher level 
of interest in the given learning topic and/or in learning through gaming than the other two 
groups. This assumption can be supported by the fact that only the results of subjects who 
used this type of support provision, thereby showing a high level of motivation, were taken 
into account. Thus, this group may have used support in a more adaptive way because its 
members wanted to master the tasks. They may also have invested more mental effort, that 
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is they may have allocated more cognitive capacity to the task and may therefore have had 
improved learning outcomes. 

In summary, educational computer games with support devices enhance learning outcomes 
compared to educational computer games without additional support. However, depen¬ 
dent on the level of learners’ interest, the type of support provision may or may not have an 
influence on learning outcomes. In order to gain a thorough understanding of the impact of 
learners’ level of interest on the effectiveness of types of support provision, more research 
needs to be conducted. In order to learn more about the effect of learners’ characteristics 
on the effectiveness of support in games, subjects need to be not only selected in such a 
way as to ensure sufficient variation in their levels of interest (e.g., in the learning topic, in 
learning with games). Also the differences between novices and experts should be studied 
since experts are better able than novices to compensate for design flaws of support devices 
and to integrate new knowledge. Such studies should be conducted with respect to novices’ 
and experts’ respective experience of virtual presence and cognitive load and to the learning 
effectiveness of different support designs. Since it was not possible to differentiate between 
the three cognitive-load types in measuring them, which is the first limitation of this study, 
future research on cognitive load needs to use instruments that distinguish the three cogni¬ 
tive load types, instruments that were not available to the researchers of the present study 
(see, for discussion, e.g. Domagk, 2009). This is not only necessary to gain insight into 
the influence of differently designed support on cognitive processes, but also to investigate 
more thoroughly whether or not cognitive load regulates the relation between support, vir¬ 
tual presence, and learning outcomes. Another limitation of this study is the relatively low 
number of participants, which may have had a negative effect on the statistical power. Thus, 
future research should use larger samples. Furthermore, the reliability of the retention and 
far-transfer tasks was slightly below the accepted consistency of measurement (i.e., Cron- 
bach’s a = .59). This seems to indicate that not all tasks assessed learners’ knowledge and 
understanding of the given physics material. Based on the fact that the average correlation 
between items for retention and transfer was as respectable (r > .300, see Field, 2009), 
however, data from both questionnaires were used. Finally, it was not possible to determine 
how frequently and for how long learners used the extrinsic support in the G_ES group. 
Therefore, it cannot be excluded that some learners did not follow the instruction to use the 
hyperlinked textbook when needed, although they said so in the questionnaire. It may be 
possible that they gathered no additional information similar to learners provided with no 
additional support. Further studies should include log-file data of the use of extrinsic sup¬ 
port (e.g., when and how learners use certain support-options during learning with games) 
combined with direct observations (e.g., eye-movement, think aloud-interviews) that could 
provide useful information on whether and how often support devices were properly used. 
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